Papers with generative LM
BeamR: Beam Reweighing with Attribute Discriminators for Controllable Text Generation (2022.findings-aacl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing have led to the availability of large pre-trained language models with rich generative capabilities. |
| Approach: | They propose a method to combine generative LMs with attribute discriminators to control different attributes of text generation. |
| Outcome: | The proposed method performs better than existing state-of-the-art approaches in sentiment steering and machine translation formality tasks. |
Beyond Memorization: The Challenge of Random Memory Access in Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Recent advances in Language Models (LMs) have shown their effectiveness in knowledge-intensive tasks. |
| Approach: | They investigate whether a generative language model is able to access its memory sequentially or randomly. |
| Outcome: | The proposed LMs are able to access memory sequentially or randomly. |
ATG: Benchmarking Automated Theorem Generation for Generative Language Models (2024.findings-naacl)
Copied to clipboard
| Challenge: | Existing generative language models (LMs) can generate new or reusable theorems, but their ability to generate new theorels is under-explored. |
| Approach: | They propose to use Metamath library to generate new theorems that can be saved as reusable knowledge for future theoretical proving. |
| Outcome: | The proposed benchmark evaluates whether an agent can generate valuable (and possibly brand new) theorems that are applicable for downstream theoretic proving as reusable knowledge. |
TRIGO: Benchmarking Formal Mathematical Proof Reduction for Generative Language Models (2023.emnlp-main)
Copied to clipboard
Jing Xiong, Jianhao Shen, Ye Yuan, Haiming Wang, Yichun Yin, Zhengying Liu, Lin Li, Zhijiang Guo, Qingxing Cao, Yinya Huang, Chuanyang Zheng, Xiaodan Liang, Ming Zhang, Qun Liu
| Challenge: | Automated theorem proving (ATP) benchmarks focus on symbolic inference but rarely involve understanding complex number combination reasoning. |
| Approach: | They propose a benchmark that requires a model to reduce a trigonometric expression with step-by-step proof and evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
| Outcome: | The proposed benchmark evaluates a generative LM’s reasoning ability on formulas and ability to manipulate, group, and factor number terms. |
An Empirical Comparison of LM-based Question and Answer Generation Methods (2023.findings-acl)
Copied to clipboard
| Challenge: | Question and answer generation (QAG) is a task of generating question-answer pairs given a context. |
| Approach: | They propose to leverage sequence-to-sequence language model fine-tuning to generate question-answer pairs given a context. |
| Outcome: | The proposed model outperforms other more convoluted approaches in the end-to-end model and is computationally light at both training and inference times. |